Based on causal theory:

hypotheses: a statement about what we should expect to observe if a causal claim is true. (Also could call this empirical prediction)

Claim:

Higher rates of economic growth cause democratization to be more likely

Claim:

Higher rates of firearms ownership cause crime to be less likely

Claim:

Lower rates of economic growth cause civil war to be more likely

Claim:

Higher rates of Nick Cage films cause a higher rate of drowning deaths?

Maybe we've been lead astray.

Claim:

X causes Y.

Implies:

If X does not happen then Y would not happen

1. If economy had grown less (more), countries would have been less (more) democratic
2. If there were more (fewer) guns, there would be less (more) crime.
3. If there were more (less) economic growth, civil would be less (more) likely
4. If Nick Cage were in fewer (more) films, fewer (more) people would die by drowning

Claim

X causes Y

Implies

If claim is true, then in one case with some level of X and some level of Y:

If that case had a different level of X, then the level of Y would have been different.

1. We can't make the same country have less economic growth and see if it doesn't democratize
2. We can't make the US gun ownership rate not change in the past and see if crime stays high
3. We can't make the same country have more economic growth to see if it prevents civil war
4. We can't unmake (or unsee) Ghost Rider 2 to see if fewer people would have died in 2012

We never can observe a case under the counterfactual condition: we can only observe a case under one (the factual) condition.

We want to test whether \(X\) causes higher \(Y\).

For sake of simplicity, imagine a situation with \(5\) cases.

Each observation is indexed by \(i\).

\(X_i:\) value of cause \(X\) for case \(i\). \(1\) for cause is present, \(0\) for cause is absent

\(Y_i:\) The outcome (dependent variable) for \(i\)

Fundamental Problem of Causal Inference:

• We don't know what cases would have looked like if they had the other value of \(X\).
• That is: we don't know what they would look like under the counterfactual.

Counterfactual causality means that every case has a value for the outcome that it would take under the factual and counterfactual conditions. We call these potential outcomes

We want to test whether \(X\) causes higher \(Y\).

For sake of simplicity, imagine a situation with \(5\) cases.

Each observation is indexed by \(i\).

\(X_i:\) value of cause \(X\) for case \(i\). \(1\) for cause is present, \(0\) for cause is absent

\(Y_i^0:\) The outcome (dependent variable) for \(i\) when \(X_i = 0\)

\(Y_i^1:\) The outcome (dependent variable) for \(i\) when \(X_i = 1\)

Our test: compare \(Y\) for \(X = 1\) and \(X = 0\)

We found that \(Y\) was \(\frac{11}{3}\) higher when \(X = 1\)

Truth: \(Y_i^1 - Y_i^0\)

The truth is the average difference for each case between condtion with \(X\) and condition without \(X\)

Truth is that there is no difference

fundamental problem of causal inference:

• We can't trust comparing higher/lower levels of X and Y
• We can't observe cases in counterfactual condition

So, we still have fundamental problem of causal inference.

Can we learn anything about causality?

Claim: \(X\) is cause of \(Y\)

\(A,B,C\) are other possible causes of \(Y\).

Problem!

Problem!

If causal claim that \(X \rightarrow Y\), then:

We generate empirical prediction:

If we observe two cases to be the same in all relevant respects except for value of \(X\), then we should observe that the two cases differ in the value of \(Y\)

This empirical prediction based on a causal claim is called comparative method or method of difference (via John Stuart Mill)

What are the relevant similarities? How many things need to be the same? How do we identify them?

1. Society is complex, humans and our institutions have lots of possible attributes, exposure to lots of different causes
2. We might not know all possible causes
3. Very difficult to actually measure all of these causes
4. Even if we did, might not find case that is exactly the same

Claim:

\(X \rightarrow Y\)

but also the case that \(\lbrace W_1, W_2, \ldots, W_\infty \rbrace \rightarrow Y\)

if true

1. We can't observe all \(W_1, W_2, \ldots, W_\infty\)
2. Unlikely to find cases that are the same on all \(\mathbf{W}\)

The fundamental problem of causal inference states we can't observe both \(Y_i^1\) (outcome when exposed to cause) and and \(Y_i^0\) (outcome when not exposed to cause) for a given \(i\).

But we can get around this if we look at many cases \(i \in \lbrace 1 \ldots n \rbrace\) and take the average.

We cannot know each individual causal effect: \(\tau_i = Y_i^1 - Y_i^0\)

But the average causal effect is the average of all individual causal effects:

\(ACE = \frac{1}{n}\sum\limits_i^n{\tau_i}\)

So it is also equal to this:

\(ACE = \frac{1}{n}\sum\limits_i^n (Y_i^1 - Y_i^0)\)

Which means it can be the difference between the averages of both potential outcomes:

\(ACE = \Big( \frac{1}{n}\sum\limits_i^n Y_i^1 \Big) - \Big( \frac{1}{n}\sum\limits_i^n Y_i^0 \Big)\)

But does this help us?

\(ACE = \Big( \frac{1}{n}\sum\limits_i^n Y_i^1 \Big) - \Big( \frac{1}{n}\sum\limits_i^n Y_i^0 \Big)\)

• \(\frac{1}{n}\sum\limits_i^n Y_i^1\) is the average over all cases of the potential outcome when the cause is present.
• \(\frac{1}{n}\sum\limits_i^n Y_i^0\) is the average over all cases of the potential outcome when the cause is absent.
• The FPIC means that for any case, we only observe \(Y_i^1\) or \(Y_i^0\), not both.

At best we can observe:

1. \(\frac{1}{n}\sum\limits_i^n (Y_i^1 | X_i = 1)\): The average of \(Y\) when \(X\) is present for the cases where \(X\) is actually present
2. \(\frac{1}{n}\sum\limits_i^n (Y_i^0 | X_i = 0)\): The average of \(Y\) when \(X\) is absent for the cases where \(X\) is actually absent

And we know from above that this can go wrong:

Take average for \(X = 1\) and for \(X = 0\) and take the difference:

1. \[ \frac{6 + 8 + 6}{3} - \frac{2 + 4}{2}\]

2. \[ \frac{20}{3} - \frac{6}{2}\]

3. \[ \frac{20}{3} - \frac{9}{3} = \frac{11}{3}\]

True effect is \(0\) !

1. If the average outcome for cases with cause \(X\) were similar to what the average outcome would be in the presence of \(X\) for cases without \(X\).
2. If the average outcome for cases without \(X\) were similar to what the average outcome would be in the absence of \(X\) for cases with \(X\)

Then:

• \(\frac{1}{n}\sum\limits_i^n (Y_i^1|X_i = 1) = \frac{1}{n}\sum\limits_i^n (Y_i^1)\)
• \(\frac{1}{n}\sum\limits_i^n (Y_i^0|X_i = 0) = \frac{1}{n}\sum\limits_i^n (Y_i^0)\)

That'd be great!

But we don't know the counterfactual outcomes, so how can we ensure they are similar?

Wait…

Didn't we just see a problem like this?

• We had a population we want to describe, but it was too large to observe every case
• We wanted a sample that looked like the population

Random Sampling

A procedure that let us get a sample such that:

• Mean of sample is unbiased estimate of mean of population

Testing causal theories

We need to generate emprirical predictions or hypotheses about what we will observe if the causal theory is correct

Testing causal theories

Causality implies a counterfactual.

Empirical Predictions for causal theories are that:

1. When we change the level of the independent variable (cause) for a specific case
2. The change in the level of the independent variable results in change in dependent variable (outcome)

Testing causal theories

But we can't change the level of the independent variable for a case:

Fundamental Problem of Causal Inference: key part empirical prediction about causal theory is never observable

Randomly assigning exposure to cause

"Treatment" and "Control" group have same potential outcomes

because they are two random samples from the same population

• Treatment group factual outcomes are same as counterfactual outcomes for Control
• Control group factual outcomes are same as counterfactual outcomes for Treatment

Best way to understand is to do it!

We have this table of potential outcomes:

Could be: effect of campaign ad on campaign contributions:

What the true average causal effect?

\[\frac{4 + 4 + 4 + 4}{4} = 4\]

Let's run a randomized experiment:

• We will put 2 people in treatment (see the campaign ad)
• We will put 2 people in control (don't see the ad)

When we do this at random:

• we are choosing one possible partition of the sample
• there are several possible partitions into Treatment/Control
• each possible partition is equally likely

Write down all possible treatment groups of size \(2\) (e.g. (1,2))

All possible treatment groups

Calculate the average \(Y^1\) ($ with ad) for every possible treatment (ad) group

Calculate the average \(Y^0\) ($ with ad) for every corresponding control (no ad) group

Calculate the average causal effect for every possible experiment (($) ad - ($) no ad) or \(Y^1 - Y^0\)

What is the average effect across all possible experiments?

\[\frac{6 + 5 + 4 + 4 + 3 + 2}{6} = 4\]

Why does this work?

If you wanted:

• you can compare \(Y^1\) for the treatment groups and \(Y^1\) for the control to see if they are the same
• you can compare \(Y^0\) for the control groups and \(Y^0\) for the treatment to see if they are the same

Average of \(Y^1\) for treated groups (factual outcome): 7.5

Average of \(Y^1\) for control groups (counterfactual outcome): 7.5

Average of \(Y^0\) for control groups (factual outcome): 3.5

Average of \(Y^0\) for treated groups (counterfactual outcome): 3.5

Internal Validity

A study has internal validity when the causal effect of \(X\) on \(Y\) it finds is not biased (systematically incorrect).

Threats to internal validity

1. Selection Bias
2. Non-excludability
3. Systematic Measurement Error

Selection Bias:

selection bias: When cases that receive a "treatment" or "cause" have different potential outcomes from those that do not.

Selection Bias:

Consider this experiment:

• We want to evaluate the effect of emergency housing assistance on homelessness prevention.
• We give assistance to the first 100 people to show up and compare them to the second 100 people.

Problem?

• People with greater need OR better resources/capabilities show up first
• Different needs or different resources probably related to propensity to become homeless
• Exposure to treatment related to other factors that might affect outcome!

Non-excludability

non-excludability occurs when the treatment in an experiment bundles multiple different treatments

Non-excludability

Does an apple a day keep the doctor away?

• Some people assigned to eat apples, some are not.
• To ensure people eat apples, we remind them every day to eat apples and give them a financial incentive to eat it.

Problem?

• Are the effects we see due to apples? To being reminded every day? To being paid?
• Lots of differences between treatment and control.

Systematic Measurement Error

systematic measurement error: error produced when our measurement procedure obtains scores that are, on average, too high or too low.

Systematic Measurement Error

We randomly assign some people to be shamed into voting (we remind them of their past voting record)

We measure whether people exposed to this treatment have higher self-reported voting rates

Problem?

• If we see a difference between treatment and control, is it due to actual differences in voting?
• Or is it due to people who are shamed self-reporting differently?
• Treatment affects results of measurement procedures \(\rightarrow\) measurement bias
• Only a problem when measurement bias is related to treatment

Randomization of cause

• Allows us to actually observe a counter-factual
• Directly tests causal theory
  • Allows us to see whether the change in \(Y\) due to \(X\) is in the right direction
  • Allows us to see whether \(Y\) changes due to \(X\) relative to the counterfactual.

Problems are solvable

• Selection bias, non-excludability, measurement bias all solvable
• Better design of experiment

External Validity

external validity is the degree to which the causal relationship we find in a study matches the causal relationship and the context identified in a causal theory

• Study has external validity if the relationship found can generalize to all the cases to which our causal theory applies
  • If our study suffers from sampling bias, then our study may lack external validity
• Study has external validity if the cause in the study is the same as the cause in our causal theory
  • If our independent variable/cause in the study lacks validity then our study may lack external validity

Broockman and Kalla (2015): Does perspective taking change minds about minority groups?

Sample

• Invite tens of thousands of people to join a survey
• Of the ~1500 that respond, randomly assign in two groups

Treatment

• One group gets canvasser for transgender rights
• Other gets canvasser for recycling

Outcome

• Survey responses on attitudes about transgender people, their rights

External Validity:

Does exposure to partisan media change people's political attitudes and make them more extreme? (E.g. Fox News, Breitbart)

Sample: Randomly chosen group of US adults

Treatment:

• Randomly exposed to 2 hours of partisan news or non-partisan movie

Outcome:

• Survey questions about political leaning, attitudes about key political issues

External Validity:

Treatment:

• Randomly exposed to 2 hours of partisan news or non-partisan movie

Problem:

• Is this short treatment the same as years of exposure on cable TV or internet news consumption?
• If we find no effect of 2 hours of watching, does this mean that emergence of partisan news didn't cause change in Americans' political attitudes?
• Cause we can manipulate in experiment not the same as cause in causal theory

Always a Trade-off:

More internal validity (unbiased calculation of causal effect) comes at the cost in external validity (relevance of study sample or cause to the causal theory)

Why?

Many relevant contexts for causal theories and interesting causes cannot or should not be manipulated at random:

• What causes democratization?
• What causes war or ethnic violence?
• What causes economic growth?
• Why were civil rights extended to oppressed minority groups?

We can't use experiments to text many causal theories

What approaches can we use instead?

1. Comparative Method
2. Correlation

Aren't they flawed?

Yes, but now that we know how and why experiments work, we can make these better.

If causal claim that \(X \rightarrow Y\), then:

We generate empirical prediction:

If we observe two cases to be the same in all relevant respects except for value of \(X\), then we should observe that the two cases differ in the value of \(Y\)

This empirical prediction based on a causal claim is called comparative method or method of difference (via John Stuart Mill)

1. We don't need cases to be identical on all attributes
2. They need to be the identical on attributes related to both cause (\(X\)) and outcome (\(Y\))
   • In short: identical on factors that affect \(Y\) and are related to or affect \(X\)

Question:

What causes states to create laws designed to exclude and dominate people based on race?

Causal claim:

Attempts to resolve conflicts within the dominant (more powerful) racial group in a society lead to the creation of a legal regime that excludes other groups.

What are other possible causes that could be related?

1. Some dominant groups more prone to in-fighting and exclusion?
   • Maybe colonizing groups. diverse backgrounds \(\rightarrow\) in-fighting
   • Colonizers \(\rightarrow\) more likely to use legal exclusion
2. Some relationships between dominant and oppressed groups more prone to in-fighting and exclusion?
   • Slavery \(\rightarrow\) conflicting interests within dominant group (owners, non-owners)
   • Slavery \(\rightarrow\) provides a framework to continue legal discrimination in the future

What are other possible causes that could be related?

1. Legal exclusion more popular at some times rather than others:
   • Racial exclusion easily justified in some eras
   • Exclusion not seen as acceptable today, so other methods used

Comparison

Background

19th Century London saw repeated outbreaks of cholera, with mass death

• Dominant view was that "miasmas" or bad air caused diseases like cholera

A novel idea…

No theory of germs, but

• John Snow, MD didn't buy into "miasmas"
• Published an essay On the Mode of Communication of Cholera in 1849
• Suggested that polluted water was means of communication

Recall:

conjunctural causation: when effect depends on combination of causes

multiple necessary conditions: effect occurs when only in presence of more than one cause

• Plant growth requires water and sunlight
• Cholera transmission requires both the bacteria and water

Comparative method

• Can help us see when there are multiple necessary conditions

Can be clue to causality

Empirical Prediction

If \(X \rightarrow Y\) or \(X\) causes \(Y\), then \(X\) and \(Y\) will be correlated

Why?

If \(X\) causes \(Y\), a shift in \(X\) implies a change in the value of \(Y\)

• So values of \(X\) and \(Y\) should move together in the direction stated by the causal theory

What is it?

correlation: an association or relationship between the values taken by two variables (X and Y)

Contrast to comparative method

correlation:

• look for patterns across many cases, rather than two or a few
• look at correlation across units at the same time
• look at correlation across time in the same units

What is it?

correlation: also has specific mathematical definition (you don't need to memorize):

\[r = \frac{\sum_{i}^n (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_i^n(x_i - \bar{x})^2}\sqrt{\sum_i^n (y_i - \bar{y})^2}}\]

What is it?

mathematically: correlation is the degree of linear association between \(X\) and \(Y\)

• Takes values between \(-1\) and \(1\)
• Values close to \(1\) or \(-1\) suggest high degree of linear association
• Values close to \(0\) suggest low degree of linear association
• Value of correlation does not tell us how much \(Y\) changes with \(X\)

What is it?

negative correlation: (correlation \(< 0\)) values of \(X\) and \(Y\) move in opposite direction:

• higher values of \(X\) appear with lower values of \(Y\)
• lower values of \(X\) appear with higher values of \(Y\)

positive correlation: (correlation \(> 0\)) values of \(X\) and \(Y\) move in same direction:

• higher values of \(X\) appear with higher values of \(Y\)
• lower values of \(X\) appear with lower values of \(Y\)

• It is possible to see perfect correlation but small change in \(Y\) across \(X\)

• It is possible to see low correlation but large change in \(Y\) across \(X\)

• It is possible to see perfect nonlinear relationship between \(X\) and \(Y\) with \(0\) correlation

weak correlation: values for \(X\) and \(Y\) do not cluster along line

strong correlation: values for \(X\) and \(Y\) cluster strongly along a line

strength of correlation unrelated to the slope of line describing \(X,Y\) relationship

Causal Theory

Higher gun ownership rates cause higher rates of firearm homicide

Expected correlation:

If theory is correct: we expect positive correlation between gun ownership rate and firearms homicides

1. Identify Phenomenon we want to explain
   • Why does it happen?
2. Develop causal theory
   • What is the cause and its effect?
   • What is the causal logic?
3. How would we know theory is right?
   • Formulate hypothesis about what we observe if theory is right
4. If \(X \rightarrow Y\), variables should move together
   • Cause \(X\) and outcome \(Y\) are correlated

1. Test:

   • measure \(X\) and \(Y\). Are they correlated?
   • If yes: evidence for theory
   • If no: evidence against theory

Identify a Phenomenon:

Identify a Phenomenon:

Why did US violent crime rates spike and then decline from the 1970s-1990s?

Develop a Causal Theory

Widespread use of and then ban on leaded gasoline caused increase and decline in crime

Leaded gasoline \(\rightarrow\) Particulate lead in the air

\(\rightarrow\) children exposed to lead \(\rightarrow\) lead poisoning

\(\rightarrow\) education, aggression, inhibition problems

\(\rightarrow\) criminal behavior

Empirical Prediction

If theory is true: positive correlation between childhood lead exposure and adult criminality

If theory is true: birth cohorts with more lead exposure have higher criminality rates in adulthood

A positive correlation betwen lead exposure and crime rates

Correlation Test: across time (New Orleans)